Skip to content

fix(cluster): hash lone surrogates as the UTF-8 Node sends - #2242

Merged
PavelPashov merged 1 commit into
redis:mainfrom
JacobLey:fix/calculate-slot-lone-surrogate
Oct 22, 2026
Merged

PavelPashov merged 1 commit into
redis:mainfrom
JacobLey:fix/calculate-slot-lone-surrogate

Conversation

@JacobLey

@JacobLey JacobLey commented Oct 22, 2026 •

Copy link
Copy Markdown
Contributor

Description

Fix cluster slot calculation for string keys that contain an unpaired UTF-16 surrogate. It follows up #2172, which vendored cluster-key-slot into lib/utils/calculateSlot.ts.

Bug

toUTF8Array() encodes a lone surrogate as a 3-byte sequence (for example \udc00 becomes ED B0 80). Node never sends those bytes. When ioredis writes a string argument to the socket, Node replaces each lone surrogate with U+FFFD (EF BF BD). So the client and the server hash different bytes:

key client slot (before) slot of the bytes sent / CLUSTER KEYSLOT
"x\ud800y" (lone high) 11133 6798
"x\udc00" (lone low) 10835 13299
"abc\ud83d" (high at end) 3201 12716
"a\udc00\ud800b" (low then high) 13938 4085

The right-hand column by capturing the bytes ioredis wrote to a socket, and by running CLUSTER KEYSLOT on a live Redis 8 server (they match)

Consequence

Commands on such a key go to the wrong node, get MOVED, and are retried. That costs an extra round trip on every command for the key. Every path that computes a slot from a string goes through calculateSlot (Command#getSlot, autopipelining including key prefixes, Pipeline, ClusterSubscriberGroup), so the fix covers all of them.

Fix

In the 3-byte branch of toUTF8Array(), a code unit in the surrogate range is replaced with 0xFFFD before it is encoded. This matches Node's UTF-8 encoding exactly:

Node's UTF-8 write passes V8's kReplaceInvalidUtf8 (string_bytes.cc, v8-primitive.h).

Valid surrogate pairs are still handled by the 4-byte branch, as before. ASCII and 2-byte characters never reach the new check.

String.prototype.toWellFormed() would also work, but tsconfig.json targets lib: ES2021 and the manual test workflow still runs Node 12–18. It was also measurably slower. Median ns/op over 2M calls on Node 24:

keys before this PR toWellFormed() + before
ASCII 144.7–146.6 144.5–145.6 153.9
non-ASCII 74.8–78.8 74.6–79.0 85.3

Tests

I added cases to test/unit/calculateSlot.ts: a lone high surrogate, a lone low surrogate, a high surrogate at the end, and a low surrogate followed by a high one. Each asserts that calculateSlot(key) equals calculateSlot(Buffer.from(key)). There is also a check against the server-measured slot 13299, and a control showing that a valid pair is unchanged.


Note

Medium Risk
Changes cluster key routing for a rare edge case (malformed surrogate strings); wrong slots previously caused MOVED retries but correct routing is critical for all cluster commands.

Overview
Fixes cluster slot hashing for keys containing unpaired UTF-16 surrogates so ioredis hashes the same UTF-8 bytes Redis sees on the wire.

toUTF8Array() in calculateSlot used to UTF-8-encode lone surrogates directly; Node replaces them with U+FFFD when writing strings (and in Buffer.from). The 3-byte encoding path now maps surrogate code units to 0xFFFD before encoding, matching V8’s invalid-UTF-8 handling; valid surrogate pairs are unchanged.

New unit tests assert calculateSlot(key) matches calculateSlot(Buffer.from(key)) for several lone-surrogate cases, one live CLUSTER KEYSLOT value, and a valid emoji pair.

Reviewed by Cursor Bugbot for commit 3fcf1c8. Bugbot is set up for automated code reviews on this repo. Configure here.

@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 22, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-10-22T02:40:54.424631Z 3fcf1c8 PR opened
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@PavelPashov PavelPashov left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@JacobLey LGTM, thank you for the contribution

@PavelPashov
PavelPashov merged commit cfa464d into redis:main Oct 22, 2026
22 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants